Efficient Skyline Computation for Large Volume Data in MapReduce Utilising Multiple Reducers
نویسندگان
چکیده
A skyline query is useful for extracting a complete set of interesting tuples from a large data set according to multiple criteria. The sizes of data sets are constantly increasing and the architecture of backends are switching from single node environments to cluster oriented setups. Previous work has presented ways to run the skyline query in these setups using the MapReduce framework, but the parallel possibilities are not taken advantage of since a significant part of the query is always run serially. In this paper, we propose the novel algorithm MapReduce Grid Partitioning Multiple Reducers Skyline (MR-GPMRS) that runs the entire query in parallel. This means that MR-GPMRS scales well for large data sets and large clusters. We demonstrate this using experiments showing that MR-GPMRS runs several times faster than the alternatives for large data sets with high skyline percentages.
منابع مشابه
Efficient Skyline Computation in MapReduce
Skyline queries are useful for finding interesting tuples from a large data set according to multiple criteria. The sizes of data sets are constantly increasing and the architecture of back-ends are switching from single-node environments to non-conventional paradigms like MapReduce. Despite the usefulness of skyline queries, existing works on skyline computation in MapReduce do not take full a...
متن کاملParallel Computation of Skyline and Reverse Skyline Queries Using MapReduce
The skyline operator and its variants such as dynamic skyline and reverse skyline operators have attracted considerable attention recently due to their broad applications. However, computations of such operators are challenging today since there is an increasing trend of applications to deal with big data. For such data-intensive applications, the MapReduce framework has been widely used recent...
متن کاملSecurity and Privacy Aspects in MapReduce on Clouds: A Survey
MapReduce is a programming system for distributed processing large-scale data in an efficient and fault tolerant manner on a private, public, or hybrid cloud. MapReduce is extensively used daily around the world as an efficient distributed computation tool for a large class of problems, e.g., search, clustering, log analysis, different types of join operations, matrix multiplication, pattern ma...
متن کاملSimultaneous Processing of Multi-Skyline Queries with MapReduce
With rapid increase of the number of applications as well as the sizes of data, multi-query processing on the MapReduce framework has gained much attention. Meanwhile, there have been much interest in skyline query processing due to its power of multi-criteria decision making and analysis. Recently, there have been attempts to optimize multi-query processing in MapReduce. However, they are not ...
متن کاملAdapting Skyline Computation to the MapReduce Framework: Algorithms and Experiments
This paper addresses the problem of skyline computation under the MapReduce framework. As a parallel programming model for data-intensive computing applications, MapReduce runs on a cluster of commercial PCs with the main idea of task decomposition and result reduction. Based on different data partitioning strategies, three MapReduce style skyline computation algorithms are developed: MapReduce...
متن کامل